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R.  SCOTT  RODGERS 

Improving  Analysis:  Dealing  with 
Information  Processing  Errors 

Though  seemingly  odd  at  first  glance,  intelligence  professionals  and  mental 
health  clinicians  have  aspects  of  their  respective  crafts  in  common.  Both 
have  the  unenviable  task  of  attempting  to  establish  methods  for  the 
reliable  and  valid  classification  and  understanding  of  individuals  and 
groups  of  individuals.  Both  are  called  upon  to  predict  future  behaviors  and 
actions.  As  mental  health  professionals,  clinicians  are  called  upon  to  help 
make  decisions,  such  as  diagnoses,  treatment  selection,  outcome  assessment 
and  recidivism,  risk  assessment,  child  custody,  admissions  into  programs, 
and  hiring  selection.  Intelligence  analysts,  on  the  other  hand,  are  called 
upon  to  predict  intentions  and  motivations  of  international  actors,  future 
actions  such  as  troop  movements,  and  likely  adversary  responses  based  on 
friendly  actions. 

Assessments  and  information  obtained  from  various  other  sources  provide 
the  foundation  for  analysts’  and  clinicians’  judgments.  Both  must  sort 
information  and  decide  overall  conclusions  and  implications.  As  has  been 
demonstrated  in  a  variety  of  professional  arenas,  humans  are  not 
particularly  skilled  at  combining  various  pieces  of  information.  Attention 
needs  to  be  placed  on  understanding  how  and  why  clinicians  and  analysts 
do  not  do  well  when  making  predictions.  Two  possible  reasons  may 
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account  for  the  relative  inability  to  generate  valid  predictions:  the  basic 
problems  associated  with  the  fallible  process  of  human  information 
processing  and  the  specific  nature  of  the  endeavor  or  subject  of  interest.1 


BASIC  PROBLEMS  OF  INFORMATION  PROCESSING 

Information  can  be  extremely  ambiguous,  consisting  of  impressions  of  how 
individuals  present  themselves.  Each  new  contact  or  individual  presents  a 
wealth  of  information  of  varying  quality.  Evaluating  this  wealth  of  fuzzy 
information  places  people  into  very  fuzzy  sets.2  Clearly,  humans  cannot 
assess  all  that  is  being  presented.  They  need  to  reduce  or  simplify  the 
information  presented.  The  question  becomes  what  information  should  be 
attended  to  in  decisionmaking  and  how  should  this  information  be  used? 
Hopefully,  training  has  shown  what  information  is  most  salient  or  worthy 
of  attention  in  deliberations.  After  shrinking  the  perceptual  field  to  a 
manageable  size,  information  is  then  examined  and  decisions  are  made 
about  the  individual  or  group  in  question.  In  this  simplified  view, 
decisionmaking  involves  two  different  steps:  information  restriction  and 
selection,  and  information  aggregation  or  processing. 

Both  steps  may  fall  prey  to  what  Richards  Heuer  describes  in  his  seminal 
work,  Psychology  of  Intelligence  Analysis,  as  mental  models.  Mental  models 
are  patterns  of  expectation  and  experience  that  form  a  mind-set  which 
predisposes  analysts  to  think  in  certain  ways.  Newly  acquired  information 
is  evaluated  and  processed  through  the  existing  mental  model,  rather  than 
used  to  reassess  the  premises  of  the  model  itself.  While  mental  models  are 
necessary  and  unavoidable  due  to  the  wealth  of  information  impinging 
upon  people’s  senses,  the  disadvantage  is  that  they  can  color  perception  to 
the  extent  that  important  information  may  be  dismissed,  distorted,  or 
ignored.  Intelligence  analysts,  in  an  effort  to  improve  estimation  and 
prediction,  must  start  with  a  clear  understanding  of  the  human  mind  and 
how  it  processes  information.3 


INFORMATION  RESTRICTION  AND  SELECTION 

Although  humans  restrict  information  in  a  variety  of  ways,  the  focus  here  is 
on  three  specific  forms  that  affect  human  information  processing:  (a)  the 
tendency  to  see  patterns  where  none  exist,  (b)  the  tendency  to  seek 
confirmatory  evidence,  and  (c)  the  use  of  preconceived  biases. 

Psychologist  Thomas  Gilovich  summarized  a  wealth  of  research  on 
humans’  perceptions  of  relations  and  causes  in  everyday  life,  and  how  they 
are  very  prone  to  impute  order  to  ambiguous  information.  People  tend  to 
strive  for  predictability  in  their  world.  Gilovich  has  argued  that  all  humans 
are  predisposed  to  look  for  and  see  order  in  the  relations  among  events, 
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but  that  this  process  is  not  without  flaws.4  He  cites  several  examples  of  the 
clustering  illusion  to  support  this  claim.  When  presented  with  random 
sequences  (either  points  on  a  map  or  sequences  of  shots  made  in  a 
basketball  game  by  different  players),  people  tend  to  focus  on  clusters  of 
points,  and  infer  a  relationship  even  though  the  actual  process  is  random. 
A  common  example  of  this  inference  of  cause  is  the  gambler’s  fallacy.  This 
occurs  where  the  probability  of  a  particular  event  is  over-evaluated,  given 
independent  prior  events  (e.g.,  estimating  the  presence  of  heads  on  the  next 
coin  toss  because  the  previous  four  tosses  resulted  in  tails). 

Heuer  has  also  described  a  simplified  rule  of  thumb  commonly  used  in 
probability  estimates,  known  as  the  availability  rule.  People  use  the 
availability  rule  whenever  they  estimate  frequency  or  probability,  based  on 
how  easily  they  can  recall  or  imagine  instances  of  whatever  they  are  trying 
to  estimate.  Although  the  availability  rule  works  well,  the  ability  to  recall 
instances  of  an  event  is  influenced  by  how  recently  the  event  occurred, 
whether  there  was  personal  involvement  in  the  event,  and  whether  the 
details  associated  with  the  event  were  more  vivid  and  memorable.  For 
example.  Central  Intelligence  Agency  (CIA)  officers  acquainted  with  the 
mole  Aldrich  Ames  would  likely  perceive  a  greater  risk  of  insider  betrayal 
than  officers  who  did  not  know  anyone  involved  in  spying  for  the 
opposition.5 

Yet,  many  events  of  concern  to  intelligence  analysts  when  predicting  future 
behavior  are  so  unique  that  past  history  does  not  seem  relevant  to  the 
evaluation  (i.e.,  terrorists  using  airplanes  as  a  weapon).  In  thinking  of  such 
events,  analysts  often  construct  scenarios.  In  these  instances,  the 
plausibility  of  the  scenario  acts  as  a  clue  to  likelihood  of  the  event.  If  no 
reasonable  scenario  comes  easily  to  mind,  the  event  is  evaluated  as  highly 
unlikely.  If  a  scenario  comes  easily  to  mind,  the  event  in  question  appears 
probable.6 

In  addition  to  probability  inaccuracies,  humans  are  prone  to  self¬ 
confirmation  in  cases  where  equivocal  information  exists  or,  in  other 
words,  “we  perceive  what  we  expect  to  perceive.”7  A  wealth  of  research 
has  demonstrated  the  human  tendency  to  search  out  and  attend  only  to 
evidence  that  confirms  one’s  ideas,  beliefs,  or  hypotheses.8  The  problem 
with  the  confirmatory  tendency  is  that  only  information  supportive  of 
one’s  beliefs  is  attended  to,  even  in  the  face  of  extremely  disconfirming 
information.  Information  that  could  provide  corrective  feedback  that  one’s 
beliefs  are  in  error  is  rarely  evaluated.  This  process  of  searching  for 
confirmation  can  lead  to  some  very  inaccurate  conclusions,  and  may  lead 
to  an  increased,  perhaps  unjustified,  confidence  in  one’s  conclusions. 

Intelligence  analysts  may  also  be  prone  to  several  additional  types  of  errors 
when  evaluating  evidence,  such  as  vividness,  unsubstantiated  consistency, 
and  uncertain  accuracy.  Regarding  vividness,  information  that  is  dramatic, 
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concrete,  or  personal  can  have  a  significant  impact  on  thinking.  For  example, 
a  single,  vivid  case  may  outweigh  a  much  larger  body  of  statistical  evidence 
or  conclusions  reached  by  abstract  reasoning.  Unsubstantiated  consistency 
can  also  be  deceptive  if  the  information  on  which  it  is  based  is  highly 
correlated,  redundant,  or  comes  from  a  small  or  biased  sample.  If  an 
analyst  has  only  a  small  amount  of  information  and  cannot  determine 
representativeness,  then  confidence  in  judgments,  based  on  that  evidence, 
must  be  low  regardless  of  consistency.  Likewise,  uncertain  accuracy  can 
impact  analytical  conclusions  when  the  caveat  concerning  source  reliability 
does  not  attenuate  the  substance  of  the  information.  Pertinent  information 
from  a  suspect  source  must  be  evaluated  for  reliability.9 

Related  to  the  errors  associated  with  evaluating  evidence  is  the  tendency  to 
reify  one’s  preconceptions.  Medical  literature  has  many  examples  of  how 
clinicians  make  errors  in  their  clinical  decisionmaking  that  are  related  to 
beliefs  regarding  specific  cues.  The  tendency  is  to  over-pathologize 
clients,10  especially  when  the  cues  of  social  class,  race,  and  sex  are  present.11 

Social  psychological  research  on  how  individuals  assess  the  causality  of 
behavior  has  also  shown  a  dichotomy  between  internal  and  external 
determinants  of  human  actions.  A  fundamental  error  made  in  judging  the 
cause  of  behavior  is  to  overestimate  the  role  of  internal  factors,  and  to 
underestimate  situational  factors.  Research  has  shown  that  people  are 
more  inclined  to  infer  that  some  behavior  was  determined  by  personal 
qualities,  or  disposition,  rather  than  external  circumstances.  Intelligence 
analysts  are  familiar  with  this  dichotomy  when  they  assess  international 
actors.  Since  situational  information  is  often  incomplete  in  the  assessment 
process,  the  tendency  is  to  assume  that  an  individual’s  predisposition  will 
determine  future  action.12 

Another  error  in  determining  behavior  specific  to  analytical  judgments  is 
mirror  imaging — assuming  (consciously  or  otherwise)  that  the  other  side  is 
likely  to  act  in  a  certain  way  because  that  is  how  the  United  States  would 
act.  But  the  goal  should  be  to  see  the  options  faced  by  foreign  leaders  as 
they  themselves  see  them.  Failure  to  understand  that  others  perceive  their 
national  interests  differently  from  the  way  Americans  do  has  been  a 
consistent  source  of  estimative  error  in  U.S.  intelligence  analysis.13 

Both  clinicians  and  analysts  need  to  be  aware  of  the  limitations  of  their 
information  processing  abilities  because  inaccurate  decisions  regarding 
individuals  under  study  can  be  easily  made.  Clinicians  or  analysts  find  it 
difficult  to  claim  that  they  are  attending  to  the  appropriate  information 
and,  if  so  doing,  that  it  is  evaluated  in  a  manner  free  from  inaccurate, 
preconceived  biases.  These  biases  could  be  related  to  the  resulting  poor 
predictions.  As  psychologist  Paul  Meehl  has  noted,  clinicians  are  not 
especially  skilled  at  selecting  the  best  information  to  which  to  attend.14 
The  same  applies  to  intelligence  analysts. 
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SPECIFIC  NATURE  OF  THE  ENDEAVOR 

Once  analysts  believe  they  have  quality  or  accurate  information,  their  next 
task  is  to  combine  the  information  and  to  establish  data  relationships. 
Evidence— data  that  is  relevant — must  be  weighted,  and  judgments  made 
as  to  which  alternative  conclusions  are  supported  by  the  evidence.15  The 
relative  validity  of  information  is  often  overlooked.  For  clinicians 
(particularly  psychologists),  usage  of  standardized  assessment  devices  helps 
obviate  some  of  the  issues  related  to  information  selection  and 
restriction.16  For  intelligence  analysts,  formal  quantitative  processes,  such 
as  Bayesian  Analysis,  do  exist,  but  seem  to  be  rarely  used.  In  most  cases, 
the  weighting  of  evidence  for  analysts  is  a  qualitative  judgment.17  The 
process  of  making  judgments  and  forming  conclusions  based  on  clinical 
experience  and  findings  can  be  seen  as  similar  to  the  process  many 
intelligence  analysts  engage  in  when  they  reach  their  judgments. 


INFORMATION  AGGREGATION 

Years  ago,  Meehl  penned  what  he  calls  his  “disturbing  little  book,” 
comparing  clinical  versus  actuarial  prediction.  In  his  review  of  the  existing 
literature,  he  documented  the  relative  superiority  of  actuarial  methods  (i.e., 
those  methods  using  population  base  rates  and/or  regression  techniques) 
over  clinical  methods  in  clinical  decisionmaking.18  But  he  held  out  hope 
for  clinical  decisionmaking  with  respect  to  situations  that  require  unique 
combinations  of  data  and  the  formulation  of  original  hypotheses.19  Meehl 
subsequently  retracted  much  of  these  hopes  for  the  clinician.20  An 
extensive  review  of  the  literature  has  demonstrated  that  time  and  time 
again,  actuarial  methods  of  clinical  prediction  surpass  clinical  methods.  In 
almost  all  cases,  optimal  weighting  by  using  regression  methods  results  in 
superior  prediction  compared  to  clinicians’  judgments.21 

In  addition,  even  simple  summing  of  predictors  (as  opposed  to  using  more 
sophisticated  regression  equations)  outperformed  the  expert  clinicians.  If  the 
optimal  weighting  (i.e.,  regressions)  of  information  was  not  adopted,  but 
instead  a  strategy  of  simply  adding  up  the  information  was  used,  this 
procedure  still  yielded  predictions  unmatched  in  accuracy  by  the  expert 
clinicians.22 

In  a  similar  vein,  Robert  Clark  notes  in  his  book.  Intelligence  Analysis: 
Estimation  and  Prediction,  that  success  in  prediction  for  intelligence 
analysts  comes  through  a  proven  prediction  methodology,  and  a 
multidisciplinary  understanding  of  the  problem  under  investigation. 
However,  he  adds  the  disclaimer  that  intelligence  analysts  must  recognize 
that,  no  matter  how  good  the  prediction  methodology  or  how  well  it  is 
applied,  they  are  likely  to  be  wrong.  As  such,  the  value  of  the  prediction 
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model  lies  in  the  assessment  of  the  forces  (organizational,  economic,  etc.) 
which  are  shaping  the  event.  The  essential  element  in  force  analysis  is  for 
the  analyst  to  determine  which  existing  forces  are  changing,  in  what 
direction,  and  how  rapidly.  If  the  forces  are  correct,  analysts  have  served 
the  policymakers,  who  will  likely  make  their  own  predictions.23 

As  Ephraim  Kam  highlights  in  his  book  Surprise  Attack:  The  Victim’s 
Perspective,  the  most  severe  error,  and  the  most  relevant  to  surprise  attack, 
is  the  assumption  that  a  trend  will  continue  in  a  straight  line.  A  successful 
prediction  should  be  able  to  forecast  potential  sources  of  change,  and  the 
timing  of  the  change.  While  psychological  difficulties  of  anticipating 
change  are  significant,  many  additional  kinds  of  uncertainty  in  military 
and  international  relations  are  available.  Many  events  are  unique  and 
devoid  of  precedent.  Therefore,  prediction  based  on  observed  regularities 
becomes  more  difficult.  The  number  of  variables  to  be  considered  in 
making  a  prediction  is  large,  and  information  is  often  incomplete  or 
misleading.  Unlike  many  fields,  reducing  this  level  of  uncertainty  by  means 
of  tests  or  experiments  is  extremely  difficult,  if  not  impossible.  While 
methods  such  as  generating  alternate  scenarios  do  exist,  they  cannot 
produce  the  clear-cut  predictions  required.24 

Early  critics  of  these  conclusions  regarding  actuarial  methods  noted  that 
the  research  failed  to  consider  (a)  the  experience  level  of  clinicians  (expert 
clinicians  may  be  much  better  in  predicting  than  are  general  clinicians,  and 
certainly  than  graduate  student  clinicians);  (b)  clinician  confidence 
(clinicians  may  be  more  confident  in  some  predications  than  others);  (c) 
that  the  situation  is  too  artificial  for  the  results  to  apply  to  clinical 
assessment  in  the  real  world;  and  (d)  the  lack  of  generalization  of 
regression  weights  obtained  in  one  study  (the  optimal  weighting  for  one 
sample  may  not  at  all  match  that  for  another  sample).25  Subsequent 
research  has  focused  on  addressing  these  concerns  within  the  literature. 

The  Impact  of  Expertise 

In  reviewing  a  wealth  of  studies,  psychologists  David  Faust,  Robyn  Dawes, 
and  Paul  Meehl  concluded  that  expertise  has  little  effect  on  the  results.  In 
conditions  where  clinicians  can  choose  information  and  collect  it  in  their 
preferred  manner,  expert  clinicians  still  performed  no  better,  and  typically 
worse,  than  actuarial  methods.  Clearly,  expert  clinicians  did  at  times 
outperform  straight  statistical  models,  but  little  consistency  was  evident. 
Expert  clinicians  evidenced  little  agreement  among  themselves  in  the 
predictions  made,  and  even  for  individual  expert  clinicians,  little  consistency 
in  prediction  accuracy  could  be  demonstrated  from  one  case  to  the  next.26 

Intelligence  professionals  emphasize  that  the  expert  perceives  his  or  her 
own  judgmental  process  as  considerably  more  complex  as  is  in  fact  the 
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case.  Experts  overestimate  the  importance  of  factors  that  have  only  a  minor 
impact  on  their  judgments,  and  underestimate  the  extent  to  which  their 
decisions  are  based  on  a  few  major  variables.  Mental  models  are  simpler 
than  they  think.  The  intelligence  analyst,  like  the  clinician,  is  typically 
unaware  of  what  variables  should  have  the  greatest  influence,  as  well  as 
which  variables  actually  have  the  greatest  influence.27 

The  Role  of  Confidence 

Similar  results  were  obtained  when  clinician  confidence  in  prediction  was 
taken  into  account.  Indeed,  many  studies  have  found  that  expert  clinicians 
often  are  no  more  accurate  than  less  expert  clinicians,  but  that  they  do 
have  greater  confidence  in  their  predictions.28  Psychologist  Lawrence 
Goldberg  concluded  that,  in  general,  no  relation  can  be  determined 
between  confidence  in  the  accuracy  of  one’s  prediction  and  its  actual 
accuracy.  Confidence  is  related  to  accuracy  only  when  the  assessment  is 
based  on  validated  procedures.29  Few  areas  of  research  in  the  field  of 
psychology  have  yielded  results  as  unequivocal  as  these.30  But  this  body  of 
literature  has  been  viewed  as  having  a  negligible  impact  on  both 
practitioners  and  the  field.31 

In  the  area  of  intelligence,  Ephraim  Kam  noted  that  overconfidence  is  a 
factor  in  surprise  attack,  among  both  analysts  and  decisionmakers.  The 
implications  for  overconfidence  can  be  viewed  on  both  the  conceptual  level 
and  the  operational  level.  Conceptually,  if  an  analyst  is  confident  that  war 
is  unlikely,  signals  and  indicators  to  the  contrary  will  not  be  easily 
recognized.  The  higher  the  degree  of  confidence,  the  more  slowly 
discrepant  information  is  acknowledged.  There  is  also  less  willingness  to 
accept  alternate  hypotheses  about  enemy  intentions.  Operationally,  the 
more  confidence  a  nation  has  that  no  attack  is  likely,  the  less  willing  it  is 
to  take  precautions  and  countermeasures  against  such  a  possibility.32 

Goldberg  noted  that  the  superiority  of  actuarial  aggregation  is  related  to 
five  issues.33  First,  clinicians  do  not  do  well  because  they  ignore  the 
different  validities  of  the  predictors.  Usage  of  sound,  actuarially  based  data 
with  uniformly  high  validity  should  obviate  this  problem.  Second, 
combining  variables  if  they  have  different  metrics  is  difficult  (e.g.,  how 
does  a  clinician  intuitively  combine  scores  from  two  variables,  one  with 
scores  ranging  from  0  to  100  and  another  with  scores  from  1  to  5?).  As  an 
example  of  obviating  this  issue,  psychological  test  data  provides 
information  that  is  normed  and  scored  in  a  common  standardized  metric 
(e.g.,  a  z  score.  A  z  score  is  a  statistical  measure  that  quantifies  the 
distance,  measured  in  standard  deviations;  a  data  point  is  from  the  mean 
of  a  data  set).  Third,  clinicians  typically  are  not  consistent  in  their 
application  of  predictions  made  from  data;  they  apply  inconsistent  weights 


INTERNATIONAL  JOURNAL  OF  INTELLIGENCE 


7 


IMPROVING  ANALYSIS:  DEALING  WITH  INFORMATION  PROCESSING  ERRORS 


629 


to  the  predictors.  For  example,  a  clinician  may  weight  one  predictor  scale 
highly  for  one  case  and  in  the  next  case  a  different  scale  is  weighted  highly. 
Clearly,  applying  a  consistent  manner  of  combining  the  data  would 
improve  prediction.  Robyn  Dawes  has  demonstrated  that  even  the  simple 
averaging  of  information  or  scales  is  superior  to  an  inconsistent  clinician 
combination  of  information.34  Fourth,  clinicians  are  insensitive  to  different 
degrees  of  redundancy  in  information.  If  added  information  is  sought  to 
improve  a  clinical  decision,  instruments  with  little  overlap  to  the  current 
measures  should  be  used.  Only  by  adding  nonredundant  information  will 
the  incremental  validity — a  prediction  above  and  beyond  that  already 
obtained— improve.  Finally,  clinicians  are  relatively  insensitive  to 
regression  effects,  and  thus  need  to  take  these  into  account  when 
interpreting  psychological  test  information. 

Clinicians  are  not  alone  in  their  relative  inability  to  outperform  actuarial 
prediction.  Identical  results  have  been  yielded  in  a  variety  of  professional 
domains  such  as  medical  diagnosis,35  predicting  bank  failures36  and  stock 
market  fluctuations,  internship  matching,37  and  predicting  a  student’s 
success  in  graduate  school.38 

HEURISTICS  IN  CLINICAL  JUDGMENT39 

Amos  Tversky  and  Daniel  Kahneman  presented  three  heuristics,  i.e.,  factors 
or  conditions  that  humans  use  to  aid  decisionmaking:  representativeness, 
availability,  and  anchoring.  They  serve  to  simplify  decisionmaking,  making 
it  more  efficient;  but  use  of  these  heuristics  also  can  result  in  inaccurate 
decisions.  The  application  of  these  heuristics  has  been  summarized  widely  in 
clinical  literature  as  they  pertain  to  both  general  human  decisionmaking40 
and  clinical  decisionmaking.41 

The  use  of  heuristics  is  also  relevant  to  intelligence  analysis.  Analysts 
regularly  assess  probabilities  with  respect  to  the  outcome  of  events,  enemy 
capabilities  and  intentions,  the  meaning  of  warning  indicators,  and  the 
credibility  of  sources.  Evaluation  is  complicated  by  numerous  bits  of 
information  of  differing  reliability  which  are  related  to  several  potential 
outcomes  of  varying  probability.42 

Represen  tativeness 

Representativeness  refers  to  the  extent  to  which  something  matches  relevant 
categories.  A  frequent  example  in  clinical  practice  is  the  comparison  of  a 
specific  client  with  diagnostic  categories.  The  clinician  observes  client 
behavior,  then  assesses  the  extent  to  which  that  behavior  fits  different 
diagnostic  types.  If  the  behavior  ox  symptoms  are  seen  as  similar  or 
representative  of  a  particular  diagnostic  category,  that  diagnosis  is 
typically  made.  In  the  intelligence  field,  as  Ephraim  Kam  has  observed, 


AND  COUNTERINTELLIGENCE  VOLUME  19,  NUMBER  4 


8 


630 


R.  SCOTT  RODGERS 


“this  heuristic  enables  one  to  estimate  the  likelihood  of  one  state  of  affairs, 
given  the  knowledge  of  another  state  of  affairs,  by  judging  the  similarity 
between  the  two.”43  The  problem  with  this  decisionmaking  heuristic  is  that 
other  relevant  information  is  often  ignored.  Three  common  problems  with 
representativeness  are:  (a)  insensitivity  to  prior  probabilities;  (b)  sample 
size;  and  (c)  predictability. 

(a)  Insensitivity  to  prior  probabilities  refers  to  the  common  failure  to  take 
account  of  base  rates  in  assessing  representativeness.  Base  rates  help 
determine  the  prior  odds  or  prior  probabilities  of  something  occurring.44 
An  example  of  insensitivity  to  base  rates  is  the  number  of  diagnoses  of 
multiple  personalities  made  by  some  clinicians.  The  actual  number  of 
individuals  with  this  diagnosis  is  extremely  rare  in  the  population,  yet 
some  clinicians  have  claimed  upwards  of  ten  such  clients  in  their  caseload. 
Besides  the  obvious  exaggeration  of  symptoms  necessary  to  make  such  a 
diagnosis,  this  assessment  ignores  the  very  rare  probability  of  occurrence 
(based  on  base  rate  data).  The  clinician  sensitive  to  base  rates  would  very 
closely  scrutinize  any  such  low  base-rate  diagnosis. 

Similarly,  in  assessing  a  situation,  an  intelligence  analyst  may  have  two 
kinds  of  evidence — specifics  about  the  event  being  examined,  and  numerical 
data  summarizing  information  about  many  similar  cases.  The  numerical 
information  is  the  base  rate  or  prior  probability.  For  example,  an  enemy 
aircraft  makes  a  nonfatal  strafing  attack.  Both  Cambodian  and  Vietnamese 
jets  are  known  to  operate  in  the  area.  The  base  rate  is  85  percent  for 
Vietnamese  aircraft  and  15  percent  for  Cambodian.  But,  a  U.S.  pilot,  with 
an  80  percent  accuracy  rating,  identified  the  aircraft  as  Cambodian.  The 
probability  that  the  fighter  was  Cambodian  is  often  based  on  the  pilot’s 
report.  But  the  greater  likelihood  is  that  the  plane  was  Vietnamese  due  to 
the  base  rate  information,  despite  the  pilot’s  identification  45 

(b)  Insensitivity  to  sample  size  refers  to  the  frequent  equating  of 
information  generated  from  large  and  small  samples.  Obviously, 
comparing  an  individual  instance  to  a  category  generated  from  a  large 
sample  is  superior  to  comparing  it  to  a  category  generated  from  a  small 
sample,  but  this  is  frequently  ignored.  Humans  manifest  this  insensitivity 
in  two  ways:  by  over-generalizing  from  their  own  limited  experience,  and 
by  over-generalizing  from  limited  observation.  Clinicians  build  their 
clinical/observational  experience  from  a  small  sample  of  the  individuals 
they  have  personally  interviewed  (or  studied),  yet  they  frequently  err  in 
valuing  their  own  sample  as  much  as  some  larger  sample.  The  favoring 
of  the  limited  personal  sample,  while  ignoring  the  information  generated 
from  larger  samples,  demonstrates  this  insensitivity  to  sample  size.  The 
other  manifestation  of  this  insensitivity  is  over-generalizing  from  a 
limited  sample.  Clinicians  and  analysts  frequently  make  decisions  from 
very  limited  bits  of  information,  and  may  be  prone  to  ignore  input  from 
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other  sources  that  have  much  more  information  developed  over  a 
longer  time. 

(c)  Insensitivity  to  predictability  is  similar  to  insensitivity  to  base  rate,  in 
that  no  account  is  taken  of  the  probability  of  events.  Where  insensitivity  to 
prior  probabilities  refers  to  the  ignoring  of  base  rate  information, 
insensitivity  to  predictability  refers  to  ignoring  the  differential  probabilities 
of  future  behavior.  Some  behaviors  and  events  are  much  more  likely  to 
occur  than  others.  Insensitivity  to  predictability  refers  to  the  common 
pattern  of  viewing  all  predictions  as  equally  likely,  or  underestimating  the 
relative  differences  in  predictability.  Predicting  a  highly  probable  event 
(e.g.,  what  the  individual  or  group  will  be  doing  tomorrow)  is  sometimes 
viewed  as  equal  in  predictability  as  is  an  improbable  event  (e.g.,  what  the 
individual  or  group  will  be  doing  next  year).  Being  able  to  accurately 
predict  tomorrow  may  lead  to  the  common  (but  inaccurate)  conclusion 
that  one  can  predict  next  year  as  well. 

A  related  concept  to  insensitivity  of  predictability  is  the  common 
misunderstanding  regarding  regression  to  the  mean.  Less  probable  states 
are  followed  by  more  probable  states.  The  most  probable  future  event 
after  an  extreme  event  is  one  that  is  less  extreme.  Predicting  that  any 
extremely  depressed  client  will  not  feel  as  depressed  in  the  next  session  is 
much  more  likely  than  that  the  client  will  become  more  depressed. 

The  problems  associated  with  the  representativeness  heuristic  have 
important  implications  for  intelligence  analysis.  Analysts  may  neglect  base 
rate  data  even  if  it  is  available.  Analysts  often  receive  limited  warning 
indicators  from  small  portions  of  enemy  behavior.  Unique  events  or 
extreme  indicators  may  be  overemphasized.  A  single  piece  of  information 
may  cause  a  false  alarm  even  when  most  of  the  earlier  indicators  do  not 
suggest  a  threat.  Analysts  are  prone  to  believe  that  small  samples  are  as 
good  as  large.  Consequently,  analysts  can  become  overconfident  of 
conclusions  coming  from  a  small  body  of  evidence,  neglecting  the 
possibility  that  this  evidence  is  not  representative.46 

Availability 

The  second  heuristic  used  by  human  information  processors,  as  discussed  by 
Amos  Tversky  and  Daniel  Kahneman,  is  that  of  availability,  which  refers  to 
the  incomplete  nature  of  our  memory  search  for  information.  To  facilitate 
the  speed  of  memory  search,  people  focus  on  only  the  most  salient  aspects, 
and  frequently  ignore  other  aspects  that  may  also  be  relevant.  Those 
aspects  that  are  more  easily  brought  to  mind  are  viewed  as  thus  more 
salient.  Availability  thus  refers  to  memory  access,  which  is  affected  by  such 
things  as  exposure,  mood,  imaginability,  and  category  vividness. 

The  bias  of  exposure  is  one  especially  relevant  to  clinicians.  Clinicians  use 
their  past  and  current  clients  as  comparisons,  thus  the  quality  of  any  decision 
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rests  upon  the  completeness  of  this  sample  and  the  clinician’s  ability  to  access 
it  completely.  Peter  Cohen  and  Jacob  Cohen  demonstrated  that  clinical 
samples  are  extremely  biased  and  unrepresentative.  The  clinical  caseload 
very  quickly  gets  filled  with  a  relatively  few  clients  who  tend  to  be  fairly 
pathological.  Given  a  familiarity  with  them,  these  individuals  are  most 
easily  accessed  as  a  comparison  group.  Because  of  the  ease  of 
retrievability,  these  few  clients  then  serve  as  an  inappropriate  basis  of 
clinical  comparison  for  decisionmaking.  Cohen  and  Cohen  called  this 
natural  tendency  to  inappropriately  make  decisions  based  on  this  very 
flawed  sample  the  “clinician’s  illusion.”47 

An  individual’s  access  to  memory  is  also  affected  by  his/her  mood.  The 
literature  on  state-dependent  learning  and  recall  provides  examples  of  this 
mood  availability  heuristic.48  Similar  to  clients  who  are  able  to  access  only 
negative  life  experiences  when  depressed,  clinicians  suffer  from  the  same 
retrievability  flaw.49  For  example,  a  clinician  who  is  feeling  angry  with  a 
client  is  more  likely  to  access  past  clients  to  whom  he/she  had  similar 
reactions.  This  access  to  past  clients  can  be  helpful,  but  conversely  the 
clinician  would  be  less  likely  to  remember  other  clients  toward  whom 
he/she  was  not  angry,  and  thus  perhaps  miss  important  comparisons. 

Biases  of  imaginability  refer  to  the  tendency  to  retrieve  information  that  is 
plausible  without  regard  to  its  probability.  People  regularly  construct  a  series 
of  possible  behaviors  or  plans  based,  to  a  large  extent,  on  their  ability  to 
imagine  their  occurring.  By  imagining  a  particular  course  of  events,  the 
likelihood  is  that  they  will  plan  accordingly,  regardless  of  the  probability 
of  these  events  transpiring.  Imaginability  becomes  a  flawed  indicator  of 
probability  of  occurrence.  Being  able  to  imagine  that  a  client  could  commit 
suicide  greatly  increases  a  clinician’s  assessment  that  it  would  occur  even 
though  it  may  be  extremely  unlikely.  Because  clinicians  incorrectly  inflate 
the  probability  of  events  due  to  their  imaginability,  they  often  take  a  very 
conservative  approach  toward  prevention,  even  in  the  face  of  highly 
unlikely  events. 

Finally,  the  availability  heuristic  of  category  vividness  also  serves  people 
well  as  information  processors  but  can  inflict  bias  in  their  decisionmaking. 
Humans  tend  to  retrieve  those  categories  that  are  most  vivid.  Aspects  that 
are  most  memorable  in  their  extremeness  and  characteristics  are  the  most 
easily  retrieved.  Information  that  is  less  exciting  or  remarkable  tends  to  be 
the  last  retrieved.  With  respect  to  clinical  decisionmaking,  this  aspect  of 
availability  ensures  that  the  past  clients  most  likely  to  be  retrieved  for 
comparison  are  those  most  disturbed,  troubling,  or  conversely,  most 
successful.  The  norm  is  much  less  frequently  accessed  because  it  is  less 
vivid.  Also,  the  tendency  is  to  be  better  able  to  access  information  that  is 
more  abstract  than  specific.  Clinicians,  for  example,  are  more  likely  to 
remember  that  a  client  has  relationship  problems  but  may  be  unable  to 
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provide  the  specifics  of  the  difficulties.  So  with  respect  to  the  availability 
heuristic,  they  are  most  prone  to  retrieve  information  that  is  vivid  (often 
defined  as  extreme),  abstract  (having  few  specifics  to  substantiate  the 
concept),  based  on  their  own  flawed  sample,  and  similar  to  their  current 
mood. 

The  Intelligence  Relationship.  Both  Richards  Heuer  and  Ephraim  Kam 
explore  aspects  of  the  availability  rules  as  they  may  apply  to  intelligence 
analysts.  Heuer,  a  former  CIA  analyst,  has  addressed  the  error  of  the 
vividness  criterion,  stating  that  a  single  vivid  case  may  negate  a  much 
larger  group  of  indicators.  Kam  points  to  the  vital  role  of  experience  as  a 
determinant  of  perceived  risk.  If  experiences  are  biased,  perceptions  are 
also  likely  to  be  inaccurate.  But,  Heuer  states  that  analysts  may  be  less 
influenced  than  others  by  the  availability  heuristic.  Intelligence  analysts 
generally  evaluate  all  available  information  and  do  not  typically  make 
quick  and  easy  references.50 

Anchoring 

The  final  major  heuristic  noted  by  Tversky  and  Kahneman  is  that  of 
anchoring,  which  refers  to  the  tendency  to  let  initial  information  and 
impressions  determine  subsequent  decisionmaking.  Even  when  presented 
with  very  different  information,  humans  seldom  shift  their  decisions  much 
from  their  initial  starting  point,  or  anchor.  For  example,  if  a  clinician 
receives  early  information  from  the  client,  or  other  sources,  that  the 
diagnosis  of  borderline  personality  disorder  may  be  appropriate,  less 
pathological  diagnoses  may  never  be  examined.  Clinicians,  too,  tend  to 
make  decisions  rapidly,  and  maintain  these  decisions  over  time.51 
Providing  more  information  to  clinicians  does  not  help  alter  incorrect 
clinical  decisions52  or  lead  to  better  decisions,  although  it  often  does  lead 
to  the  false  impression  that  the  better  decision  has  been  made.53 

The  implications  of  anchoring  impressions  to  intelligence  analysis  are 
clear.  Periodically,  analysts  must  update  estimates,  either  because  of  a 
changing  situation  or  because  of  important  new  information.  A  natural 
starting  point  for  such  an  adjustment  is  the  initial  definition  of  the 
situation  or  the  previous  estimate  on  that  subject.  But  the  anchoring 
heuristic  suggests  that  analysts  may  not  sufficiently  change  their  estimates. 
As  Kam  has  stated,  “Their  judgment  is  affected  by  the  initial  anchor,  and 
their  review  of  estimates  may  lag  behind  their  receipt  of  incoming 
information  and  perception  of  changing  situations.”54 

The  heuristics  of  representativeness,  availability,  and  anchoring  are 
important  aids  in  human  decisionmaking  in  that  they  allow  for  the 
efficient  processing  of  information.  But  each  heuristic  carries  with  it  a  bias 
that  can  affect  the  quality  of  the  decisions  made.  Both  analysts  and 
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clinicians  can  rely  too  much  on  memory  and  their  own  idiosyncratic 
weighting  of  information.  Actuarial  models  do  not  rely  on  memory,  and 
can  be  combined  in  a  variety  of  straightforward  manners,  even  by  just 
averaging  the  different  scales.55 


TRAINING  AND  EXPERIENCE  AS  REMEDIES  TO  POOR  JUDGMENT? 

Perhaps  a  major  reason  for  the  failure  of  clinicians  and  intelligence  analysts 
to  learn  from  their  experience  is  the  hindsight  bias — the  tendency  to  falsely 
believe  that  they  are  able  to  accurately  predict  an  event  after  the  event  has 
transpired.56  A  common  sports  page  term  applied  to  this  bias  is  “Monday 
morning  quarterbacking,”  wherein  the  wisdom  of  certain  game  plays  or 
strategies  that  occurred  the  previous  day  can  be  criticized,  with  the  claim 
of  certainty  that  the  critic  would  have  done  otherwise  had  he/she  but  had 
the  chance.  This  bias  has  been  documented  repeatedly,  and  may  help 
account  for  the  continued  failure  to  use  additional  information.57  This 
hindsight  bias  creates  what  personality  psychologist  H.  J.  Einhorn  calls  an 
“illusion  of  learning.”58 

Given  that  humans  as  processors  of  information  are  quite  fallible,  what  is 
to  be  done?  Should  all  attempts  at  decisionmaking  be  abandoned  in  favor  of 
statistical  models?  Clearly,  statistical  models  have  more  predictive  accuracy 
than  clinical  or  analytical  judgment.59  The  response  to  this  question  is  a 
qualified  no.  Clinicians  and  analysts  do  have  the  ability  to  observe  and 
select  relevant  information. 

Comprehending  the  nature  of  information  processing  has  significant 
implications  for  intelligence  analysis,  however.  The  circumstances  under 
which  accurate  perception  is  the  most  difficult  are  usually  the  circumstances 
under  which  intelligence  analysis  is  conducted — highly  ambiguous 
situations  due  to  limited  information;  bits  and  pieces  of  information  which 
must  be  processed  incrementally;  and  pressure  for  early  judgments  and 
conclusions.  Richards  Heuer  concludes:  “That  intelligence  analysts  perform 
as  well  as  they  do  is  testimony  to  their  generally  sound  judgment,  training 
and  dedication  in  performing  a  dauntingly  difficult  task.”60 

The  problems  outlined  here  have  implications  for  the  management  as  well 
as  conduct  of  intelligence  analysis.  Given  the  difficulties  inherent  in  the 
human  processing  of  complex  information,  a  prudent  management  system 
should:  (a)  encourage  products  that  clearly  outline  their  assumptions  and 
chains  of  inference  and  that  specify  the  degree  of  source  unreliability;  (b) 
support  analyses  that  periodically  review  key  problems  from  the  ground 
up,  in  order  to  avoid  pitfalls  of  the  incremental  approach;  (c)  emphasize 
procedures  that  elaborate  alternative  hypotheses.  The  ideal  is  to  generate  a 
full  set  of  hypotheses,  systematically  evaluate  them  individually,  and 
identify  the  one  that  is  the  best  fit  for  all  the  data.  And  (d),  educate 
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consumers  about  the  limitations,  as  well  as  capabilities,  of  intelligence  in 
order  to  define  a  realistic  set  of  expectations.61 


DECISIONMAKING  AIDS 

Given  the  limitations  in  information  processing,  several  recommendations 

are  offered.62 

1.  Adopt  a  scientific  approach  to  information  evaluation  and  hypothesis  testing. 
This  includes  not  confusing  the  ability  to  explain  with  the  ability  to 
predict.  Analysts  should  focus  on  making  explicit  predictions,  then 
assess  the  extent  to  which  these  predictions  are  borne  out.  This  process 
of  making  predictions,  and  assessing  their  outcomes,  forces  analysts  to 
be  explicit  about  assumptions  and  helps  mitigate  the  “hindsight”  bias.63 

2.  Get  quality  information.  Robyn  M.  Dawes  has  noted  that  clinicians 
typically  get  poor  feedback  information.  Even  if  appropriate  hypothesis 
testing  was  conducted,  the  quality  of  information  obtained  has  provided 
little  corrective  feedback.64  For  example,  rarely  do  clinicians  obtain 
information  on  what  has  transpired  with  their  clients  after  termination 
of  service.  Frequently,  the  only  cases  where  feedback  is  obtained  are 
those  that  have  not  succeeded  and  the  patients  return  for  treatment. 
Attempts  should  be  made  to  obtain  reliable  and  valid  information 
following  termination  to  evaluate  the  accuracy  of  any  predictions.  Also, 
care  should  be  taken  in  using  the  intelligence  customer’s  acceptance  of 
intelligence  products  as  accurate  or  good  feedback.  In  psychology  this  is 
commonly  called  the  P.  T.  Barnum  effect — the  tendency  for  people  to 
accept  very  general  or  vague  characterizations  of  themselves  or  events 
and  take  them  to  be  accurate. 

3.  Think  Bayesian.  Thomas  Bayes,  a  Presbyterian  minister  and  British 
mathematician  (1702-1761),  urged  that  probabilities  be  revised  when 
more  is  learned  about  an  event.  For  analysts,  this  means  being  aware  of 
base  rates,  as  they  are  related  to  the  probability  of  occurrence  of 
different  behaviors,  and  the  probability  of  predictability  of  different, 
future  behaviors.  Thinking  Bayesian  requires  attention  to  the  full  range 
of  individuals,  both  with  and  without  the  characteristics  of  focus.  The 
ability  to  think  Bayesian  requires  knowledge  of  simple  Bayesian 
probability  rules,  but  it  also  requires  extensive  knowledge  of  population 
probabilities.65  The  development  of  appropriate  databases  could  help 
provide  some  of  the  information  on  base  rates  and  help  determine  the 
predictability  of  some  behaviors. 

4.  Consider  alternative  hypotheses  and  engage  in  disconfirming  hypothesis 
testing.  As  noted,  humans  tend  to  seek  confirming  evidence,  but  this 
strategy  is  not  beneficial  for  accuracy  of  decisionmaking.  The  best 
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solution  is  to  specify  disconfirming  evidence,  then  seek  out  this 
information. 

All  human  beings  make  various  errors  of  attribution  and  inference  in  their 
information  processing.  By  extension  analysts  will  tend  to  make  the  same 
errors.  But,  validated,  objective  measures  and  actuarial  processes  provide 
an  avenue  to  improve  the  accuracy  of  decisions.  Yet  care  must  still  be 
taken  in  their  development,  validation,  selection,  and  interpretation.  To 
continue  to  ignore  the  findings  discussed  here  and  to  not  develop  the 
databases  and  processes  necessary  to  mitigate  information  processing 
errors  means  that  both  clinicians  and  analysts  will  forever  be  considered 
poor  predictors  of  future  actions. 
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